Automatic Reduction of the Appearance of Mach Band Effect on Radiographs

ABSTRACT

Disclosed techniques include radiograph analysis based on Mach band effects. A dental radiographic image is obtained. A high contrast structure on the image is detected. The high contrast structure comprises a radiopaque structure. The structure is analyzed to identify a dental restoration. Contrast is evaluated for the presence of Mach band effects from the dental restoration. A boundary for the dental restoration is determined based on the analyzing and the evaluating. The image is colorized to display the dental restoration based on the boundary that was determined wherein the color for the dental restoration is represented as a darker color than originally on the image.

This application claims priority to provisional application 63/128,879,filed Dec. 22, 2020, which is herein incorporated by reference in itsentirety.

TECHNICAL FIELD

This application relates generally to analysis and more particularly toautomatic reduction of the appearance of Mach band effect onradiographs.

BACKGROUND ART

Regular dental office visits enable dentists to diagnose and treatdental disease in its earliest stages. Dentists can identify dentalcaries (for example, cavities), which are small holes that form due totooth decay. Cavities form in places such as in grooves, between teeth,and at the margins of dental work. When cavities are identified, adentist will remove all the tooth decay and then fill the cavity with arestoration (for example, a filling), which can be made of materialssuch as composites, gold, and ceramics. To find problems which cannot beseen through a visual examination, dentists use radiographs (forexample, x-rays). In dental x-rays, grayscale images of teeth, bone, andthe tissues around them are examined. Cavities are seen as radiolucent(e.g., dark areas and the like) on teeth. X-rays can show dark areaswhich are not cavities but are related to an optical illusion called theMach band effect. This occurs at the edge of two areas with significantdifference in their brightness (e.g., one area can be dark, the otherbright and the like). Because restorations (such as filling, crown, rootcanal treatment or any other restorative operation) are made of densematerials, they appear as high contrast, radiopaque structures (e.g.,bright areas and the like) in x-rays. The gray areas of a tooth, likeenamel and dentin, can appear darker near the edges of restorations dueto the Mach band effect. Such optical illusion can lead to misdiagnosisthat the darker appearing areas may be diagnosed as cavities. This maylead to unnecessary dental treatment. Conversely, misdiagnosing a cavityas a Mach band leads to skipping needed treatment.

SUMMARY OF THE EMBODIMENTS

In accordance with one embodiment of the invention, acomputer-implemented method for mitigating a Mach band effect in adigitized radiographic image. The method is performed by a computersystem executing computer processes comprising:

-   -   using a computer vision system to analyze the image to identify        a boundary between two regions associated with a set of        physiological features and having a contrast difference        sufficient to warrant an inference of presence of the Mach band        effect therein; and    -   modifying the image to reduce the inferred Mach band effect.

In an optional embodiment, the physiological feature of interest is adental restoration. Optionally, the computer vision system is configuredto differentiate between a dental restoration and a naturally occurringanatomical feature. Also optionally, the computer vision system is aneural network, and as a further option a convolutional neural networkor transformer neural network.

Optionally, when using a neural network, said neural network is trainedusing a plurality of images, a first subset of which includes images ofdental restorations. Further optionally, the neural network is trainedusing a second subset of the plurality of images which include imagesonly of naturally occurring anatomical features.

Further optionally, the first subset and second subset are annotatedbefore the neural network is trained.

In an optional embodiment, upon identifying the boundary a delta regionis added beyond an edge of the dental restoration shown in the image.Also optionally, using the computer vision system to analyze the imagefurther comprises identifying a decay. Optionally, the identified decayis adjacent to the dental restoration. Further optionally, after thecomputer vision system analyzes the image to identify decay, calculatinga probability metric for cavity existence within the image.

Optionally, the dental restorations comprise a filling. Also optionally,the computer implemented method further comprises causing display of thedental restoration and the naturally occurring anatomical feature indistinctive colors. Further optionally, the display of decay is shown ina distinct color from the dental restoration and the naturally occurringanatomical feature.

Optionally, the color of the dental restoration and the naturallyoccurring anatomical feature are based on a user selection. Alsooptionally, the method generates a dental treatment based on theidentified decay. Further optionally the computer vision system is usedto analyze the image based on radiolucencies in the image.

Optionally, the radiographic image includes radiographic data. Alsooptionally, the computer vision system further comprises analyzingfurther images associated with the image to identify dental restorationand Mach band effects. Optionally, said further images are taken over aperiod of time.

In accordance with another embodiment of the invention a non-transitorystorage medium is provided. The storage medium stores instructions that,when executed, establish computer processes, the computer processescomprising:

-   -   using a computer vision system to analyze the image to identify        a boundary between two regions associated with a set of        physiological features and having a contrast difference        sufficient to warrant an inference of presence of the Mach band        effect therein; and    -   modifying the image to reduce the inferred Mach band effect.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of embodiments will be more readily understood byreference to the following detailed description, taken with reference tothe accompanying drawings, in which:

The following detailed description of certain embodiments may beunderstood by reference to the following figures wherein:

FIG. 1 is a flow diagram of processes for radiograph analysis based onMach band effects in accordance with one embodiment of the invention.

FIG. 2 is a flow diagram of processes for structure colorization inaccordance with one embodiment of the invention.

FIG. 3 is a block diagram representation of a system for high contraststructure analysis in accordance with an embodiment of the invention.

FIGS. 4A and 4B are two radiographic images with a boundary taken by theimager 322 of FIG. 3.

FIG. 4C is a radiographic image with dental restorations 402 taken bythe imager 322 in FIG. 3.

FIG. 5 is a block diagram representation of a neural network inaccordance with one embodiment of the invention.

FIG. 6 is a block diagram of a system for radiograph analysis 600, wherethe radiographic analysis includes reduction of Mach band effects inaccordance with an embodiment of the invention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Definitions. As used in this description and the accompanying claims,the following terms shall have the meanings indicated, unless thecontext otherwise requires:

A “set” is a group comprising at least one member.

A “computer vision system” is an image processing arrangement that mayemploy traditional image processing, or convolutional neural networkprocessing, or transformer neural network processing, or any othermethod of image processing.

A “computer process” is the performance of a described function in acomputer system using computer hardware (such as a processor,field-programmable gate array or other electronic combinatorial logic,or similar device), which may be operating under control of software orfirmware or a combination of any of these or operating outside controlof any of the foregoing. All or part of the described function may beperformed by active or passive electronic components, such astransistors or resistors. In using the term “computer process” we do notnecessarily require a schedulable entity, or operation of a computerprogram or a part thereof, although, in some embodiments, a computerprocess may be implemented by such a schedulable entity, or operation ofa computer program or a part thereof. Furthermore, unless the contextotherwise requires, a “process” may be implemented using more than oneprocessor or more than one (single- or multi-processor) computer.

A “color in an image” is a visual attribute, in the image, selected fromthe group consisting of: a grayscale value in a grayscale image and acolor value in a color image.

Techniques for radiography analysis based on Mach band effects aredisclosed. A dental radiographic image can be obtained. A high contraststructure on the image can be detected. The high contrast structurecomprises a radiopaque structure. The structure can be analyzed toidentify a dental restoration. The dental restoration comprises afilling. The analysis differentiates between dental restorations andnaturally occurring anatomical features. This analysis can beaccomplished by using traditional computer vision methods or deep neuralnetworks. First, restorations from an image are extracted usingaforementioned methods. Then, areas near the restorations can beanalyzed for potential Mach band effects. Based on the analyzing therestorative structures, a boundary for the dental restorations aredetermined and filled with a darker color. Determining the boundarycomprises adding a delta region beyond an edge of the dental restorationshown in the image. The dental restoration can be shown as a firstcolor. Anatomical tissue is shown as a second color. Decay is shown as athird color. The analyzing comprises identifying decay.

Various features, aspects, and advantages of various embodiments willbecome more apparent from the following further description.

In the disclosed materials, understanding Mach band effects and theirimpact on radiographs helps in the analysis of x-rays. The radiographscan include dental x-rays. Mach band effects are optical illusions thatcan enhance the contrast at the borders or edges of slightly differingshades of gray that are in contact with each other. The optical illusioncauses the darker shade of gray to appear lighter at the edge, while thelighter shade of gray appears darker at the edge. In the context ofobserving dental radiographic images, the Mach band effects can cause ahuman to distort the border of dental restorations, such as a fillings,crowns, and bridges, and the adjacent naturally occurring anatomicalfeatures associated with a tooth.

Due to the Mach band effects, a decay can appear as an optical illusion.A practitioner can determine whether decay is present by seekingabnormal variances in density within the tooth, however, the Mach bandeffects can lead a practitioner to conclude a false positive for decay.The false positive results from difficulty in distinguishing a Mach bandeffect from actual decay. The clear determination of the boundarybetween the dental restoration and the tooth is necessary to determinethe presence or absence of decay adjacent to the dental restoration. Thedisclosed techniques can be used for other anatomical portions beyondmouth and dental regions. X-rays of various joints and body portions canbe similarly analyzed. Resulting radiographs can be colorized. Colorscan be changed for plates, pins, artificial joints, and other devicesinserted into a body.

Proposed techniques include adding a delta region to the border toextend it slightly beyond the edge of the dental restoration. The deltaregion is an incremental expansion of the region associated with thedental restoration. The delta region, therefore, expands the borderbetween the dental restoration and the tooth. The expansion can beaccomplished by adding a set of pixels around the area of therestoration. In some embodiments, the expansion can be accomplished bygrowing the restoration area by fixed distance or a fixed percentage.

In one embodiment, a dental restoration can be colorized to have therestoration displayed with a darker color than originally included inthe image. The darker color can be grayscale or it can be a color suchas dark brown, dark blue, or some other color.

In one embodiment, the restoration replacement color is calculated asthe average color of the immediate surrounding tooth area. The averagecolor of an area is the sum of each RGB value across all pixels withinthe area divided by the number of pixels in the area. The tooth area isdetermined by a tooth instance segmentation neural network algorithmusing modeling in a manner analogous to the model described above inconnection with tooth decay. Such a neural network may, for example, bea convolutional neural network (CNN) or a transformer neural network. Insome embodiments, the tooth area is determined by other methods, such ascomputer vision (CV). The immediate surrounding tooth area is the subsetof the tooth pixels located within a short distance from the restorationarea.

In a similar embodiment, instead of replacing the entire restorationarea with a solid color, the brightness of the restoration area isautomatically reduced to more closely match the average brightness ofits immediate surrounding tooth area. The benefit over single colorreplacement is that any texture is preserved.

In one embodiment, an adjustment knob is used to allow the operator tomanually decrease the brightness of a restoration area. The knob can bea software graphical interface or a hardware input device. The knoballows the operator to tune, in small increments, the brightness levelof a restoration area to reduce the Mach band effect. The operator canperform a reset to restore the restoration area to its originalbrightness.

The modified dental restoration can reduce any optical illusion causedby Mach bands around the restoration. Color adjustment can be performedautomatically using standard colors or using user selected colors. Aplurality of colors can be used to highlight critical aspects of thedental radiographic image including the dental restoration, anatomicaltissue, the border between the dental restoration and the tissue, decay,and so on. In various embodiments, features such as analyzing thestructure to identify a dental restoration, evaluating contrast forpresence of Mach band effects from the dental restoration, determiningthe boundary of the dental restoration, and colorization of the dentalrestoration are accomplished using deep learning or traditional computervision methods.

In one embodiment, a dental radiographic image is obtained from anindividual. The radiographic image can include a high contraststructure, where the high contrast structure can include a dentalrestoration. Radiographic data is obtained using ionizing andnonionizing radiographic techniques. The obtaining can include obtainingfurther radiographic images of the individual. A high contrast orradiopaque structure is detected on the image. The high contraststructure can include a shape, a point, a curve, and so on. The highcontrast structure can be partially or fully included within the image.The structure that was detected can be analyzed to identify a dentalrestoration. The dental restoration can include a filling, a crown, abridge, an implant, or other dental restoration. Mach band effects fromthe dental restoration can be evaluated. The Mach bands, which can beperceived in the region of a boundary, particularly at an abrupt change,may be present within the dental radiographic image. A boundary isdetermined for the dental restoration based on analyzing the structureand evaluating the contrast for potential Mach band effects.

FIG. 1 is a flow diagram of processes for radiograph analysis based onMach band effects in accordance with one embodiment of the invention.The flow diagram 100 is based on a computer-implemented method foranalysis, wherein the analysis can be performed on dental radiographicimages. The analysis is based on Mach band effects. The flow diagram 100includes obtaining, at step 110, a dental radiographic image, such asshown in 410 in FIG. 4, discussed below. In various embodiments, theradiographic image is based on ionizing and nonionizing radiation. Insome embodiments, the dental radiographic image includes a dental x-rayimage. The dental radiographic image can be uploaded by a user, obtainedfrom a repository of radiographic images such as a HIPAA compliantrepository, downloaded over a computer network, and so on. In someembodiments, the image includes radiographic data. The flow diagram 100includes detecting, at step 120, a high contrast structure on the imagebased on one or more processing techniques, such as image processingtechniques. Image processing techniques include edge detection, pixelvalue variations or ranges, and so on. The high contrast structure caninclude an anatomical tissue feature such as tooth, bone, dentalrestoration, implant, or appliance, and the like. In some embodiments,the high contrast structure includes a radiopaque structure.

The flow diagram 100 includes analyzing, at step 130, the structure toidentify a dental restoration based on various attributes in the imagesuch as density of the high-density structure, shape of the structure,location of the structure within the image, and so on. Analyzing thestructure differentiates between dental restorations and naturallyoccurring anatomical features. Dental restorations include fillings,crowns, implants, appliances such as bridges, and the like. Analyzing,in step 130, is based on radiolucencies in the image. In one embodiment,the analyzing in step 132 is accomplished using computer visionalgorithms. Computer vision gives computers have a high levelunderstanding of videos or images. These methods exclude deep learningmethods. In Computer vision methods, algorithms with fixed or adaptiveparameters help accomplish a task such as finding edges in an image, orfixing objects within images. Moreover, these methods are incorporatedwith deep learning methods such as fine-tuning the results of deeplearning models using some image statistics. These algorithms may or maynot require a training step prior to analysis purpose.

In the flow diagram 100, analyzing is performed, in step 134, using aconvolutional neural network (CNN). In other embodiments, the sameprocess is performed using different forms of computer vision, such astransformer neural networks. The embodiment of flow diagram 100 uses aCNN, which is a configuration of a neural network that is well suited toimage analysis applications. The CNN includes a plurality of layers,where a subset of the layers can include feature learning layers, and asecond subset of the layers include classification layers. The layerswithin the CNN can include one or more of: a convolution layer, arectifier linear unit (ReLU) layer, a pooling layer, a batchnormalization layer, a flattening layer, a fully connected layer, asoftmax layer, etc. The CNN is used to detect restoration instances andcan create a segmentation mask which assigns a value to each pixel. Thisway, each restoration can be a separate object.

In flow diagram 100, the CNN is trained, at step 136, using a pluralityof images, a subset of which includes dental restorations. Training of aneural network, a deep learning network, a CNN, etc. is accomplished byproviding a training dataset which includes images and annotationresults based on those images. In some embodiments, the annotations arepolygons delineating the restoration areas. The results may include thata dental restoration is present within an image known to include adental restoration. Other results can include that no dental restorationis present within an image known to not include a dental restoration. Byapplying a sufficiently large, typically having one thousand to ahundred thousand images, training dataset to the CNN, the CNN “learns”to identify which images include a dental restoration and which imagesdo not. In one embodiment, the CNN learns to identify preciserestoration areas at the pixel level. It solves a problem known asinstance segmentation where each restoration area within an image is an“instance.” The accuracy of the CNN is improved by applying moretraining datasets to the training of the CNN. In flow diagram 100, theCNN is trained, at step 138, using a further subset of the plurality ofimages that include only naturally occurring anatomical features.

In some embodiments, the CNN or other computer vision (CV) detect decayson radiographic images that are darker regions compared to restorations.The CNN and CV algorithms can be trained to segment out or createbounding boxes around decays. In other embodiments, a combination of CNNand CV algorithms is used to detect decays. In some embodiments, CNN andCV algorithms are used to detect other dental diseases or anomalies onradiographic images such as wear, erosion, broken teeth, remainingpieces of broken teeth and so on. These diseases or anomalies havesimilar structures as decays where they are typically darker compared torestorations.

In some embodiments, the CNN and CV algorithms can help detect decaysand other diseases around restorations such that they would helpannotators such as hygienists or dentists to eliminate potential falsenegatives (for example, a dentist may think a decay is actually becauseof Mach band effect, but the models can help show the object is indeed adecay).

The flow diagram 100 includes evaluating, at step 140, contrast forpotential presence of Mach band effects from the dental restoration.Discussed throughout, Mach band effects are optical illusions that canenhance or alter the contrast at the borders or edges of slightlydiffering shades of gray that are in contact with each other. Theresults of the optical illusion are that the darker shade of grayappears, to a human, lighter at the edge, while the lighter shade ofgray appears darker at the edge. Flow diagram 100 includes determining aboundary 150 for the dental restoration based on analyzing the structureand evaluating the Mach band effects. The boundary is between the radioopaque high contrast structure and one or more naturally occurringanatomical structures. The boundary may include a point, a line, acurve, etc. In some embodiments, determining the boundary 150 is furtherbased on images and analyzing and evaluating the further images fordental restorations and Mach band effects. The further images includefurther dental radiographic images, where the further images can beobtained at substantially the same time as obtaining the image. In otherembodiments, determining the boundary is based on a set of images takenover a period of time. In flow diagram 100, determining the boundaryincludes adding a delta region 152 beyond an edge of the dentalrestoration shown in the image. Adding the delta region to a boundary isused to reduce Mach band effects. The flow diagram 100 further includesidentifying possible decay 154. Identifying possible decay can be basedon evaluating the dental restoration for potential Mach band effects.Detection of decay adjacent to the border of the dental restoration canbe difficult to detect to the Mach band effects. Flow diagram 100includes calculating a probability metric for cavity existence withinthe image. The metric can be based on a value, a range of values, apercentage, a probability, a text result such as “Likely” or “Notlikely,” etc.

Flow diagram 100 includes colorizing the image 160 to display the dentalrestoration, which reduces Mach band effects. In one embodiment, thedental restoration can be colorized a first color using a variety oftechniques. In various embodiments, colorizing is performed based on auser selection, the user selects a color from a pulldown menu, from acolor wheel or chart, and the like. In other embodiments, colorizing canbe accomplished automatically, based on defaults, industry standardcolor conventions, etc. In some embodiments, the image and the firstcolor comprise a grayscale monochromatic image. The first color canfurther include a grayscale selection, a fill pattern, etc. In someembodiments, the first color is black. Other features within the dentalradiographic image can be colorized. In some embodiments, anatomicaltissue can be shown as a second color, a second grayscale selection, asecond fill pattern, and the like. Further colors can be used foradditional anatomical tissue types such a tooth, gum, jaw, and so on. Insome embodiments, decay can be shown as a third color. The decay canalso be shown as a third grayscale selection, a third pattern, etc. Thecolorizing the decay can enhance viewing of the decay. The flow diagram100 further includes determining a dental treatment 170 based on theanalyzing the structure and the determining the boundary for the dentalrestoration and the colorizing the restoration to reduce Mach bandeffects. The treatment can include replacement of the dental restorationsuch as refilling dental caries, replacing or providing a dentalrestoration such as a crown, providing a dental restoration such as abridge, and the like. Various steps in the flow diagram 100 may bechanged in order, repeated, omitted, or the like without departing fromthe disclosed concepts. Various embodiments of the flow diagram 100 canbe included in a computer program product embodied in a non-transitorycomputer readable medium that includes code executable by one or moreprocessors.

FIG. 2 is a flow diagram of processes for structure colorization inaccordance with one embodiment of the invention. A dental radiographicimage can be obtained for analysis including analyzing a high contraststructure detected within the image and identifying a dentalrestoration. The dental restoration can include a filling, a crown, animplant, a bridge, an appliance, and so on. In order to aid a user, suchas a practitioner, to find decay or other dental conditions that need tobe addressed, structures, objects, regions, and so on, within the imagecan be colorized. Such colorizing enables radiograph analysis based onMach band effects. The structure is analyzed to identify a dentalrestoration and the contrast is evaluated for the potential of Mach bandeffects. A boundary for the dental restoration is determined based onthe analyzing the structure and the evaluating the Mach band effects.The dental restoration is colorized based on the determined boundary.

The flow diagram 200 includes detecting 210 a radiopaque or highcontrast structure on a radiographic image. In one embodiment, theradiographic image is an x-ray image. In radiographic images, radiopaquestructure blocks or reduces transmission of an electromagnetic or radiosource. The radio opaque structure can show up in negative within animage such as a dental radiographic image. Flow diagram 200 includesdetermining a boundary for the dental restoration based on the analyzingand the evaluating. Flow diagram 200 includes determining a color orfill pattern 220 for the radiopaque or high contrast structure, such asblack and white, a grayscale, a range of colors, and so on. The fillpattern can include lines, dots, dashes, zigzags, natural patterns, etc.In different embodiments the color is selected automatically, based on auser selection, and the like. The user can select a color from a digitalcolor wheel, a pulldown menu, etc. The fill patterns can include a colorselection for the background and the fill patterns. In some embodiments,the color can be black. Flow diagram 200 includes colorizing thestructure 230, which includes filling the structure with a color, a fillpattern, and the like. Various steps in the flow diagram 200 may bechanged in order, repeated, omitted, or the like without departing fromthe disclosed concepts. Various embodiments of the flow diagram 200 canbe included in a computer program product embodied in a non-transitorycomputer readable medium that includes code executable by one or moreprocessors.

FIG. 3 is a block diagram representation of a system for high contraststructure analysis in accordance with an embodiment of the invention.Dental radiographic images, such as x-ray dental images, include dataassociated with naturally occurring anatomical features, dentalrestorations, and so on. When present, high contrast structures in animage are analyzed to determine the presence of dental restorations,potential decay, and so on. High contrast structure analysis enablesradiograph analysis with reduced Mach band effects. If a high contraststructure is detected on the image, the structure is analyzed toidentify a dental restoration and the contrast is evaluated forpotential Mach band effects. A boundary of the dental restoration isdetermined based on analyzing the restorative structure and evaluatingthe contrast for potential Mach band effects. The dental restoration isthen colorized based on the boundary to reduce Mach band effects.

Block diagram 300 includes an input from which radiographic data can beobtained, such as a mouth 310. The data can be obtained “live” from anindividual during a dental appointment, uploaded by a user, obtainedfrom a repository such as a HIPAA compliant repository, downloaded overa computer network such as the internet, and so on. Block diagram 300includes processor 320. Processor 320 may comprise a standaloneprocessor, a server, a plurality of processors, processors withinintegrated circuits or chips, and the like. The processor 320 is used tocapture input data. In the embodiment of FIG. 3, processor 320 iscoupled to or in communication with imager 322, which is used to capturedata associated with the input 310. The imager may include aradiographic imager, where the radiographic imager can be used tocapture x-ray images of the mouth. The processor 320 is coupled to arestoration detection component 324, which is used to detect dentalrestoration such as a filling, a crown, an implant, a bridge, etc. Theprocessor 320 is also coupled to decay detection component 326, whichdetects decay in untreated teeth, decay in a tooth that was previouslytreated, and so on. In some embodiments, detecting decay includes decaythat is adjacent to the detected dental restoration.

In block diagram 300 processor 320 is coupled to a deep learningcomponent 328. The deep learning component can be used for analysis of ahigh contrast structure within the dental radiographic image. In someembodiments, analyzing based on deep learning is performed using aconvolutional neural network or a transformer neural network. In oneembodiment, the neural network is trained using image data prior toapplication of deep learning to the analysis. Processor 320 is coupledto a metric calculator 330. The metric calculator can be used tocalculate a probability metric for cavity existence within the image.The probability metric can be based on a text evaluation such as high,medium, or low probability; on a percenter; on a value, and so on. Theprocessor is coupled to a band effects evaluation component 332. TheMach band effects can appear as optical illusions and can obscure thepresence or absence of decay at a boundary of a dental restoration. Theprocessor 320 is coupled to a boundary determiner 334, which determinesa boundary based on image processing techniques such as edge detection.In some embodiments, determining the boundary includes adding a deltaregion beyond an edge of the dental restoration shown in the image.

The processor 320 is coupled to a colorizer 336. The colorizer 336 addssynthetic or “false” color to the dental radiographic image and reducesMach band effects. In one embodiment, the colorizer 336 colorizes theimage to display the dental restoration and naturally occurringanatomical features. The dental restoration can include a filling,crown, implant, etc. The anatomical tissue can include a tooth, a gum, ajaw, etc. In some embodiments, the dental restoration and a naturallyoccurring anatomical feature are displayed in distinct colors. The colorof the restoration or feature can be determined by a user selecting thecolor. In various embodiments, additional colors are used to highlight,outline, differentiate, or otherwise denote other features, objects, andso on, within the image. In some embodiments, decay is shown as a colordistinct from the anatomical feature and restoration. One or more colorsused for colorizing the dental radiographic image can be selected basedon a variety of techniques. In some embodiments, the colorizing isperformed based on a user selection such as from a pulldown menu, colorwheel such as a digital color wheel, and the like. In furtherembodiments, the colorizing is accomplished automatically, based ondefault colors, preset colors, standard colors, etc. The colors that canbe assigned to the dental restoration, decay, anatomical features, canbe changed, modified, altered, and so on. In one embodiment, the colorsare inverted within the colorizing. In some embodiments the differentcolors may be different shades of gray in a grayscale image. In otherembodiments, each color may be a different color in a color image.

FIGS. 4A and 4B are two radiographic images having a boundary, taken bythe imager 322 of FIG. 3. A radiograph image such as a dentalradiographic image may include a high contrast structure. The highcontrast structure includes a dental restoration such as a crown, afilling, an implant, and so on. A boundary is determined adjacent to adental restoration. The boundary is used in colorizing a dentalrestoration to reduce Mach band effects. The colorized radiographicimage with boundary is used for radiographic analysis with reduced Machband effects.

An x-ray image 400 is shown in FIG. 4A. An x-ray image with addedboundary 404 is shown in FIG. 4B. An original image 410, such as aradiographic image or x-ray, can include an unaltered or unmanipulatedimage. The original image can be analyzed to detect a high contraststructure in the image, where the high contrast structure can include adental restoration such as a filling, a crown, etc. Data associated withthe dental restoration is included with x-ray image 400, includingpatient data. The image shows a high contrast structure, which is adental restoration 412. In addition to the dental restoration, the image410 shows further detail 414 of the tooth to which a dental restoration(e.g., root canal treatment) was applied. A second x-ray image 420 isshown, representing an annotated version of the x-ray image 410. Theannotated image 420 shows restoration 422 representing restoration 412in image 410. A boundary 424 has been added to the image 420. Theboundary is determined and added to the image using image processingtechniques such as edge detection techniques. In one embodiment,determining the boundary includes adding a delta region to expand theboundary by an incremental amount. In some embodiments, a dentaltreatment is determined based on analyzing the structure and determiningthe boundary for the dental restoration and colorizing the dentalrestoration based on the boundary. The dental treatment may includeproviding a filling, a crown, an implant, no treatment, etc. In otherembodiments, determining the boundary can be based on additional imagesand associated analyzing and evaluating the further images for thedental restoration and Mach band effects.

FIG. 4C is a radiographic image with dental restorations 402 taken bythe imager 322 in FIG. 3. Dental radiographic images such as x-rayimages are obtained and analyzed to detect a high contrast structure inthe images. The high contrast structure includes a dental restorationsuch as a filling, a crown, an implant, etc. The radiographic data orimage data can include images of an individual prior treatment beingperformed, after treatment was performed, and so on. Dental radiographicimage 430, includes a high contrast structure. The x-ray image 402 isobtained through uploading by a user, downloading from a library ofimages, obtaining from a repository of images, and so on. The x-rayimage 402 is comprised of a bitewing image or other dental x-ray image.Analysis can be performed on the image, wherein analyzing candifferentiate between dental restorations and naturally occurringanatomical features. The x-ray image 430 includes naturally occurringanatomical features such as teeth 432, gums, bones, such as jaw bones,etc. The x-ray image 430 further includes a high-density structure 434,where the high-density structure can include a filling. In otherembodiments, the high-density structure 434 in the image is associatedwith other dental restorations such as a crown, an implant, a bridge,and the like.

FIG. 5 is a block diagram representation of a neural network inaccordance with one embodiment of the invention. In differentembodiments, the neural network 500 is a transformer neural network or aconvolutional neural network. The neural network includes, inter alia,combinations of one or more inputs, one or more layers, and one or moreoutputs. The layers comprise one or more nodes, which process data andreceive as input, inter alia, a weight and a bias. Data is processed bya given node by applying a weight, a bias, etc., associated with thenode, to the processing of the data. Nodes within a layer of a neuralnetwork are interconnected with an input, an output of one or more nodesfrom a prior layer, etc. A neural network enables radiographic analysiswith reduced Mach band effects.

Block diagram 500 shows a neural network including a plurality of layers520, 530, where the layers can perform operations associated with dentalradiographic image processing. The neural network 500 receives an inputimage 510 which is uploaded by a user, obtained from an imagerepository, downloaded over a computer network such as the internet, andso on. The neural network 500 analyzes the obtained image. Prior toimage analysis the neural network requires training. In one embodiment,the neural network is trained using a plurality of images, a subset ofwhich include dental restorations. The images comprise a trainingdataset and expected results for each image within the training dataset.In some embodiments, the subset is annotated before the neural networkis trained. In other embodiments, the neural network is trained using aplurality of images, a subset of which include dental restorations suchas crowns, fillings, bridges, implants, etc. In other embodiments, theneural network is trained using a subset of the plurality of imageswhich include only naturally occurring anatomical features. Thenaturally occurring anatomical features include enamel, dentine, a root,etc., associated with a tooth, bone, gum, etc. In one embodiment, atraining dataset includes a plurality of images, which include dentalrestorations and a plurality of images that do not include dentalrestorations. The training dataset also includes information thatidentifies the images that include dental restorations and the imagesthat do not include dental restorations. The neural network is trainedby applying the images and examining the predictions or inferences madeby the neural network and comparing them to the information associatedwith the images. The weights and biases associated with the CNN areadjusted as part of the training, to improve the success rate ofcorrectly differentiating images that contain dental restorations andimages that do not. Similar methods are used for transformer neuralnetworks The adjusting further speeds convergence by the neural networkto its result or inference.

The layers within the neural network 500 include feature learning layers520. The feature learning layers 520 are used to determine informationabout the content of the input image 510. The information can includefeatures of the image, where a feature can include an edge or a point,curves or boundaries, an object, and so on. The feature learning layers520 can include a convolution, transformer, among other things dependenton the type of neural network. The feature learning layer 520 alsoincludes a rectifier linear unit (ReLU) layer 522. The convolution andReLU perform a convolution operation and a ReLU operation. The ReLUoperation comprises an activation operation. The feature learning layersalso include pooling layer 524. A pooling operation includes “downsampling” of an image by simplifying further analysis of the image toaccomplish local translation invariance within the image. One or moreconvolution and ReLU layers and one or more pooling layers can beincluded within the feature learning layers of the neural network. Inthe example block diagram shown, a second convolution and ReLU layer 526is included and a second pooling layer 528 is included.

In addition to the feature learning layers, the neural network includesclassification layers 530. The classification layers seek to classify animage, where the classification includes identifying an image asincluding a dental restoration, identifying an image as not including adental restoration, and so on. The classification layers includeflattening layer 532, which takes, as input, an image represented by a2-D matrix. The 2-D matrix can be converted to a vector. Theclassification layers include a fully connected layer 534. As the nameimplies, each node within the connected layer is connected to each nodeof a previous layer. In the block diagram shown, each node of the fullyconnected layer is connected to each node of the flattening layer. Theclassification layers include a softmax layer 536, which is used tonormalize a probability distribution function. The softmax layerconverts a vector of real values to a vector of real values that sumto 1. Large values in the unconverted vector can receive a highprobability while near-zero or negative values can receive a lowprobability. In one embodiment, the softmax layer can be used as a finalactivation layer in the neural network. The result of processing theinput image by the neural network includes inference 540. The inference,or prediction, can include whether an image includes a dentalrestoration or not. The inference can further include a presence orabsence of decay. The inference can also include a treatment suggestion.

In some embodiments, a combination of CNN and computer vision algorithmswork together to determine features such as restorations and decays, aswell as the borders where there may potentially be an optical illusion.For example, a CNN model can detect the features and use CV algorithmsat the dark regions around the edges so the bright features can bedetected. Then another algorithm can tell whether there may be a Machband effect around the borders. In some embodiments, CV algorithms helpimprove restoration detection models (the CNN model), in a way todecrease false positives and false negatives.

FIG. 6 is a block diagram of a system for radiograph analysis 600, wherethe radiographic analysis includes reduction of Mach band effects inaccordance with an embodiment of the invention. The system 600 includesa set of processors 610 attached to memory 612, which storesinstructions. The system 600 includes a display 614 coupled to the setof processors 610 for displaying data, intermediate steps, instructions,x-ray images, Mach band effect data, and so on. In one embodiment, thememory 612 stores instructions, which when executed by the set ofprocessors 610: obtain a dental radiographic image; detect a highcontrast structure on the image; analyze the structure to identify adental restoration; evaluate contrast for potential Mach band effects;determine a boundary for the dental restoration based on the analyzingthe structure and the evaluating the Mach band effects; and colorize thedental restoration based on the boundary that was determined wherein thecolor for the dental restoration can be represented as a darker colorthan originally on the image. Analyzing the structure includesidentifying decay, wherein the decay can be adjacent to the dentalrestoration. Detecting decay is aided by determining a boundary for thedental restoration. The radiograph analysis with reduced Mach bandeffects can be accomplished using processors 610, computers, servers,remote servers, cloud-based servers, and the like.

The system 600 includes a collection of instructions and radiographicdata 620. The instructions and radiographic data 620 are stored using anelectronic storage coupled to the one or more processors, a database,one or more code libraries, precompiled code segments, source code,apps, or other suitable formats. The instructions include instructionsfor detecting a high contract structure in an image based onradiographic data analysis. In some embodiments, the radiographic dataincludes x-ray data. The instructions include instructions for analyzingthe structure to identify a dental restoration such as a filling, acrown, and so on. Instructions and radiographic data 620 includeinstructions for evaluating contrast for presence of Mach band effectsfrom the dental restoration. The Mach band effects include opticalillusions seen as bands adjacent to a dental restoration. Theinstructions 620 further determine a boundary of the dental restoration.

The system 600 includes obtaining component 630. Obtaining component 630includes functions and instructions for obtaining a dental radiographicimage. More than one dental radiographic image is obtained, where thedental images can include various types of radiographic images. Thedental radiographic images include x-ray images such as bitewing,periapical, full mouth survey, panoramic, occlusal, and so on. Thedental radiographic data is obtainable from a local database, a remote,cloud-based database, a mesh-based database; user uploads; and so on. Inoptional embodiments, the dental radiographic data is encrypted to meetsecurity and handling requirements such as Health Insurance Portabilityand Accountability Act (HIPAA) requirements. The dental radiographicdata is based on a set of images, a plurality of images taken over aperiod of time, and the like.

The system 600 includes a detecting component 640. In some embodiments,the detecting component 640 includes functions and instructions fordetecting a high contrast structure on the image. In some embodiments,the high contrast structure includes a radiopaque structure. In oneembodiment, the radio opaque structure includes an anatomical structuresuch as a bone, a tooth, and so on. Alternatively, the radio opaquestructure includes a dental restoration such as a filling, a crown, andthe like. In some embodiments, detecting the high contrast structure isbased on edge detection, by a data value of range of data values withinthe image, etc. The system 600 includes an analyzing component 650,which includes functions and instructions for identifying a dentalrestoration. Analyzing the structure is based on identifying thestructure, locating the structure, and so on. In some embodiments,analyzing differentiates between dental restorations and naturallyoccurring anatomical features. The dental restorations include afilling, a crown an implant, a bridge, etc. The naturally occurringanatomical features include portions of the tooth such as enamel,dentine, pulp cavity, root, and so on. In some embodiments, theanalyzing is accomplished using deep learning. Deep learning isaccomplished using a neural network. A neural network for deep learningis trained by providing the neural network with dental images where thecorrect inferences based on the dental images are known. Some dentalimages for training in this context include a variety of dentalrestorations while other images for training are devoid of arestoration. In the course of training for deep learning, the neuralnetwork learns to recognize dental restorations and natural features. Inother embodiments, the neural network is trained to recognize differentfeatures of an image, which may have high contrast differences with therest of the image. In some embodiments, the analyzing is performed usinga neural network such as a transformer neural network or convolutionalneural network. The neural network includes a plurality of layers. Invarious embodiments, the layers within the neural network include one ormore of convolution layers, pooling layers, flattening layers, fullingconnected layers, softmax layers, etc. In other embodiments, theanalyzing can further includes identifying decay. The decay may beadjacent to a dental restoration, remote from the dental restoration,etc. In some embodiments, analyzing is based on radiolucencies in theimage.

The system 600 includes an evaluating component 660. In someembodiments, evaluating component 660 includes functions andinstructions for evaluating contrast for presence of Mach band effectsfrom the dental restoration. In one embodiment, a Mach band effectincludes an artifact within a dental radiographic image, where theartifact can be attributable to human vision, based on luminancedetected by a retina. The Mach band can be found adjacent to a curvedsurface such as a tooth, a dental restoration, and the like. The Machband effect causes false positives for the presence of decay, masks thepresence of decay, and the like.

The system 600 includes a determining component 670. In one embodiment,determining component 670 includes functions and instructions fordetermining a boundary of the dental restoration based on analyzing thestructure and evaluating the Mach band effects. In some embodiments, theboundary is determined based on an edge detection technique. Theboundary between a dental restoration and a tooth may not be abruptlydelineated within a given dental radiographic image. Therefore, in someembodiments, after determining the boundary, a delta region is addedbeyond an edge of the dental restoration shown in the image. In otherembodiments, determining the boundary is based on analyzing andevaluating additional associated images for dental restoration and Machband effects. The further images include additional dental radiographicimages showing different angles of the dental restoration. In someembodiments, the images include dental radiographic images obtained atsubstantially the same time. In further embodiments, determining theboundary can be further based on images taken over a period of time. Theperiod of time can include one or more days, weeks, months, or years.

The system 600 includes a computer program product embodied in anon-transitory computer readable medium. In some embodiments, thecomputer program product comprises code which causes one or moreprocessors to perform operations of: obtaining a dental radiographicimage; detecting a high contrast structure on the image; analyzing thestructure to identify a dental restoration; evaluating contrast forpresence of Mach band effects from the dental restoration; determining aboundary for the dental restoration based on the analyzing the structureand the evaluating the Mach band effects; and colorizing the image todisplay the dental restoration based on the boundary that was determinedwherein the color for the dental restoration can be represented as adarker color than originally on the image.

Each of the above methods may be executed on one or more processors onone or more computer systems. Each of the above methods may beimplemented on a semiconductor chip and programmed using special purposelogic, programmable logic, and so on. Embodiments may include variousforms of distributed computing, client/server computing, and cloud-basedcomputing. Further, it will be understood that the depicted steps orboxes contained in this disclosure's flow charts are solely illustrativeand explanatory. The steps may be modified, omitted, repeated, orreordered without departing from the scope of this disclosure. Further,each step may contain one or more sub-steps. While the foregoingdrawings and description set forth functional aspects of the disclosedsystems, no particular implementation or arrangement of software and/orhardware should be inferred from these descriptions unless explicitlystated or otherwise clear from the context. All such arrangements ofsoftware and/or hardware are intended to fall within the scope of thisdisclosure.

The block diagrams and flowchart illustrations depict methods,apparatus, systems, and computer program products. The elements andcombinations of elements in the block diagrams and flow diagrams showfunctions, steps, or groups of steps of the methods, apparatus, systems,computer program products and/or computer-implemented methods. Any andall such functions—generally referred to herein as a “circuit,”“module,” or “system”—may be implemented by computer programinstructions, by special-purpose hardware-based computer systems, bycombinations of special purpose hardware and computer instructions, bycombinations of general-purpose hardware and computer instructions, andso on.

A programmable apparatus which executes any of the above-mentionedcomputer program products or computer-implemented methods may includeone or more microprocessors, microcontrollers, embeddedmicrocontrollers, programmable digital signal processors, programmabledevices, programmable gate arrays, programmable array logic, memorydevices, application specific integrated circuits, or the like. Each maybe suitably employed or configured to process computer programinstructions, execute computer logic, store computer data, and so on.

Any combination of one or more computer readable media may be utilizedincluding but not limited to: a non-transitory computer readable mediumfor storage; an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor computer readable storage medium or anysuitable combination of the foregoing; a portable computer diskette; ahard disk; a random access memory (RAM); a read-only memory (ROM), anerasable programmable read-only memory (EPROM, Flash, MRAM, FeRAM, orphase change memory); an optical fiber; a portable compact disc; anoptical storage device; a magnetic storage device; or any suitablecombination of the foregoing. In the context of this document, acomputer readable storage medium may be any tangible medium that cancontain or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

Computer program instructions may include computer executable code. Avariety of languages for expressing computer program instructions mayinclude without limitation C, C++, Java, JavaScript™, ActionScript™,assembly language, Lisp, Perl, Tcl, Python, Ruby, hardware descriptionlanguages, database programming languages, object oriented programminglanguages, functional programming languages, imperative programminglanguages, and so on. In embodiments, computer program instructions maybe stored, compiled, or interpreted to run on a computer, a programmabledata processing apparatus, a heterogeneous combination of processors orprocessor architectures, and so on. Without limitation, embodiments ofthe present invention may take the form of web-based computer software,which includes client/server software, software-as-a-service,peer-to-peer software, or the like.

In embodiments, a computer may enable execution of computer programinstructions including multiple programs or threads. The multipleprograms or threads may be processed approximately simultaneously toenhance utilization of the processor and to facilitate substantiallysimultaneous functions. By way of implementation, any and all methods,program codes, program instructions, and the like described herein maybe implemented in one or more threads which may in turn spawn otherthreads, which may themselves have priorities associated with them. Insome embodiments, a computer may process these threads based on priorityor other order.

Unless explicitly stated or otherwise clear from the context, the verbs“execute” and “process” may be used interchangeably to indicate execute,process, interpret, compile, assemble, link, load, or a combination ofthe foregoing. Therefore, embodiments that execute or process computerprogram instructions, computer-executable code, or the like may act uponthe instructions or code in any and all of the ways described. Further,the method steps shown are intended to include any suitable method ofcausing one or more parties or entities to perform the steps. Theparties performing a step, or portion of a step, need not be locatedwithin a particular geographic location or country boundary. Forinstance, if an entity located within the United States causes a methodstep, or portion thereof, to be performed outside of the United States,then the method is considered to be performed in the United States byvirtue of the causal entity.

While the invention has been disclosed in connection with preferredembodiments shown and described in detail, various modifications andimprovements thereon will become apparent to those skilled in the art.Accordingly, the foregoing examples should not limit the spirit andscope of the present invention; rather it should be understood in thebroadest sense allowable by law.

The embodiments of the invention described above are intended to bemerely exemplary; numerous variations and modifications will be apparentto those skilled in the art. All such variations and modifications areintended to be within the scope of the present invention as defined inany appended claims.

What is claimed is:
 1. A computer-implemented method for mitigating aMach band effect in a digitized radiographic image, the method performedby a computer system executing computer processes comprising: using acomputer vision system to analyze the image to identify a boundarybetween two regions associated with a set of physiological features andhaving a contrast difference sufficient to warrant an inference ofpresence of the Mach band effect therein; and modifying the image toreduce the inferred Mach band effect.
 2. The method of claim 1, whereinthe set of physiological features include a dental restoration.
 3. Themethod of claim 1, wherein the computer vision system is configured todifferentiate between a dental restoration and a naturally occurringanatomical feature.
 4. The method of claim 3, wherein the computervision system is a neural network.
 5. The method of claim 4, wherein thecomputer vision system is a convolutional neural network.
 6. The methodof claim 4, wherein the computer vision system is a transformer neuralnetwork.
 7. The method of claim 4, further comprising training theneural network using a plurality of images, a first subset of whichincludes images of dental restorations.
 8. The method of claim 7,wherein training the neural network further comprises using a secondsubset of the plurality of images which include images only of naturallyoccurring anatomical features.
 9. The method of claim 8, furthercomprising, before training the neural network, annotating the firstsubset and second subset.
 10. The method of claim 1, further comprising,upon identifying the boundary, adding a delta region beyond an edge ofthe dental restoration shown in the image.
 11. The method of claim 1,wherein using the computer vision system to analyze the image furtherincludes using the computer vision system to identify a region of decayin the image.
 12. The method of claim 11, wherein using the computervision system to identify the region of decay further includes using thecomputer vision system to identify the region of decay adjacent to adental restoration.
 13. The method of claim 11, further comprisingcalculating a probability metric for cavity existence within the image.14. The method of claim 3, wherein the dental restoration comprises afilling.
 15. The method of claim 3, further comprising causing displayof the dental restoration and the naturally occurring anatomical featurein distinctive colors.
 16. The method of claim 15, wherein causingdisplay of the dental restoration and the naturally occurring anatomicalfeature includes causing a grayscale display of the dental restorationand naturally occurring anatomical feature.
 17. The method of claim 15,further comprising causing display of decay in a first color, the dentalrestoration in a second color, the naturally occurring anatomicalfeature in a third color, wherein the first, second, and third colorsare mutually distinct.
 18. The method of claim 17, further comprisingreceiving a user selection of the first, second, and third colors andimplementing the user selection in causing display.
 19. The method ofclaim 11, further comprising causing display of a dental treatment planbased on the identified region of decay.
 20. The method of claim 1,wherein using the computer vision system to analyze the image furthercomprises analyzing radiolucencies in the image.
 21. The method of claim1, wherein the image includes radiographic data.
 22. The method of claim1, wherein using the computer vision system to identify dentalrestoration and Mach band effects further includes analyzing a set ofadditional images corresponding to the image.
 23. The method of claim22, wherein the set of additional images includes an image taken at afirst time distinct from a second time at which the digitizedradiographic image was taken.
 24. A non-transitory storage mediumstoring instructions that, when executed by a computer, establishcomputer processes, the computer processes comprising: using a computervision system to analyze the image to identify a boundary between tworegions associated with a set of physiological features and having acontrast difference sufficient to warrant an inference of presence ofthe Mach band effect therein; and modifying the image to reduce theinferred Mach band effect.